The FAUST framework: Free-form annotations on unfolding vascular structures for treatment planning

For complex interventions, such as stenting of a cerebral aneurysm, treatment planning is mandatory. Sketching can support the physician as it involves an active involvement with complex spatial relations and bears a great potential to improve communication. These sketches are employed as direct annotation on 2D medical image data and print outs, respectively. Annotating 3D planning models is more difficult due to possible occlusions of the complex spatial anatomy of vascular structures. Furthermore, the annotations should adapt accordingly to view changes and deforming structures. Therefore, we developed the FAUST framework, which allows creating 3D annotations by freely sketching in the 3D environment. Additionally to generic annotations, the physician is supported to create the most common treatment options with sketching single strokes only. We allow an interactive unfolding of vascular structures with adapting annotations to still convey their meta information. Our framework is realized on the zSpace, which combines a semi-immersive stereoscopic display and a stylus with ray-based interaction techniques. We conducted a user study with computer scientists, carried out a demo session with a neuroradiologist and assessed the performance. The user study revealed a positive rating of the interaction techniques and a high sense of presence. The neuroradiologist stated that our framework can support treatment planning and leads to a better understanding of anatomical structures. Our performance evaluation showed that our sketching approach is usable in real-time with a large number of annotations. Furthermore, our approach can be adapted to a wider range of applications including medical documentation.